Methods and apparatus for bounded image data analysis and notification mechanism

ABSTRACT

The present invention relates to a method and apparatus for preliminary optical character recognition document imaging and user notification on an electronic device with image capture capabilities. More specifically, this disclosure relates to an electronic device capable of connecting to a network, wherein a document image may be transmitting and stored securely in an external server.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the full benefit of U.S.Provisional Patent Application Ser. No. 62/032,543, filed Aug. 2, 2014,and titled “METHODS AND APPARATUS FOR BOUNDED IMAGE DATA ANALYSIS ANDNOTIFICATION MECHANISM”, the entire contents of which are incorporatedherein by reference.

FIELD OF INVENTION

This invention relates to methods and apparatus for processing an imagecaptured via an image capture device. In some embodiments, the imagecapture device may be a mobile device and a processor in the mobiledevice may perform a preliminary scan and notify the user of the scanresult. More particularly, the present invention relates to performing apreliminary bounded image data analysis on a document prior totransmission of the document to a server for continued remoteprocessing, wherein the server may perform a complete characterizationof the document and/or store the imaging results within a virtual cloudbased image storage.

BACKGROUND

With the increasing use of smart phones equipped with cameras andintegrated email and text communication platforms, people have awidespread convenience of electronically sending a picture or documentfrom almost anywhere. However, in many cases a captured image of adocument may not be legible when reprinted or processed at the messagesdestination. A sender may not realize that the document is not legibleuntil it is too late to easily resend. Traditional methods of opticalcharacter recognition (OCR) of an entire document before sending isoften impractical due to the significant processing power required tolocally OCR each document.

In such examples, the necessary information may be generally minimalwith easy-to-read fonts, wherein the image may be small and onlyspecific text may be necessary to complete the action. Currentapplications are generally not targeted to process large, complexdocuments.

What is needed therefore is an apparatus and associated methods fortimely, conveniently and efficiently capturing and notifying a userimmediately whether the document can be properly analyzed transmittingelectronically the image or document for remote processing.

SUMMARY

Accordingly, the present invention provides apparatus and methods forpreliminary bounded image data analysis and notification mechanism on anelectronic device with image capture capabilities.

The details of one or more embodiments of the invention are set forth inthe accompanying drawings and the description below. The accompanyingdrawings that are incorporated in and constitute a part of thisspecification, illustrate several embodiments of the invention and,together with the description, serve to explain the principles of theinvention: Other features, objects, and advantages of the invention willbe apparent from the description, drawings and the claims herein.

A system of one or more computers can be configured to performparticular operations or actions by virtue of having software, firmware,hardware, or a combination of them installed on the system that inoperation causes or cause the system to perform the actions. One or morecomputer programs can be configured to perform particular operations oractions by virtue of including instructions that, when executed by dataprocessing apparatus, cause the apparatus to perform the actions. Onegeneral aspect includes an image feature processing apparatus including:an image capture device for capturing a set of image data; a preliminaryimage recognition mechanism operative to identify and analyze potentialcharacteristics included in the set of captured image data, wherepreliminary image recognition includes—automated identification of afeature based upon at least partial recognition of the feature; and anotification mechanism configured to notify a user of a result of thepreliminary image recognition as compared to predefined parameters. Anotification may include no artificial delay in order to provide aprompt process. Other embodiments of this aspect include correspondingcomputer systems, apparatus, and computer programs recorded on one ormore computer storage devices, each configured to perform the actions ofthe methods.

Implementations may include one or more of the following features. Onefeature may include an example with an image feature processingapparatus where a preliminary image recognition mechanism providesoptical character recognition identifying potential text characters fromthe image data, where preliminary optical character recognition includesone or both of: partial character recognition of potential textcharacters and character recognition of a portion of at least one textcharacter. The example may also include a notification mechanism whichmay notify a user of a preliminary optical character recognition basedon predefined parameters such as, but not limited to minimal font size,percent success of optical character recognition (OCR) attempts, totaltime to attempt an OCR as specified by the user for a document typecaptured in the image data. Other examples may include an image featureprocessing apparatus which may be capable of connecting to cloud storageand processing server. The image feature processing apparatus mayinclude a mobile device. The image feature processing apparatus mayinclude a digital camera. The image feature processing apparatus mayalso be configured to identify potential text characters from a documentimage and analyze the potential text characters with optical characterrecognition to provide a degree of character recognition.

There may be methods which also include transmitting the captureddocument image to a cloud server for storage of the document image onthe cloud server. In some methods a transmission may be based on aresult of analyzing the transmitted captured document image where theresult includes a success result based upon recognition of one or bothof a partial character recognition or a recognition of a portion of atleast one text character. The method may further include steps of:transmitting the document image to an external server, where theexternal server is configured to identify the potential text charactersfrom the document image and analyze the potential text characters withoptical character recognition. The method may further include steps of:categorizing the document image based on predefined content criteria ofthe recognized text. The method may further include steps of: promptinga user to accept or reject the document image. The method may furtherinclude steps of prompting the user to input authorization information,where the authorization information is capable of granting access to asecure data storage system.

Some methods may have exemplary steps where a notification is performedand the notifying includes at least one of an audible notification, avisual notification, or a tactile notification. Methods may furtherinclude steps of: identifying borders of the document image. The methodmay also include recognizing an orientation of the at least onepotential text character. In some methods, the notification of a failmay include prompting the user to select or capture an alternatedocument image. Implementations of the described techniques may includehardware, a method or process, or computer software on acomputer-accessible medium.

One general aspect may include methods of preliminary optical characterrecognition and notification, the methods may include steps of:identifying at least one potential text character from a captureddocument image, where an electronic device with image capturecapabilities is configured to store the document image. The methods mayinclude analyzing the at least one potential text character withpreliminary optical character recognition, where the analyzing createsresults based on an ability to recognize at least one text character,and where the preliminary optical character recognition includes one orboth a partial character recognition or a character recognition of aportion of the at least one text character. Some methods may includecommunicating the results to a notification device configured to notifya user of the results based on predefined parameters of a fail result ora success result. The method of notifying a user may include at leastone of an audible notification, a visual notification, or a tactilenotification. Other aspects and examples of these methods may includecorresponding computer systems, apparatus, and computer programsrecorded on one or more computer storage devices, each configured toperform the actions of the methods.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates exemplary embodiments of engaging a Bounded ImageData Analysis (BIDA) on a local system, wherein the local systemcomprises at least an image-capturing mechanism and a processor, areillustrated for an exemplary portable computing device.

FIG. 1B illustrates exemplary embodiments of engaging a Bounded ImageData Analysis (BIDA) on a local system, wherein the local systemcomprises at least an image-capturing mechanism and a processor, areillustrated for an exemplary portable computing device.

FIG. 2A illustrates exemplary embodiments of the BIDA capture systemutilizing the image capturing mechanism on a local system.

FIG. 2B illustrates exemplary embodiments of the BIDA capture systemutilizing the image capturing mechanism on a local system.

FIG. 3A illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 3B illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 3C illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 3D illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 3E illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 3F illustrates an exemplary step for utilizing a local system withBIDA capabilities and uploading a captured document.

FIG. 4A illustrates an exemplary step for developing a document imagewith recognizable specified features through use of overlapdifferential.

FIG. 4B illustrates an exemplary step for developing a document imagewith recognizable specified features through use of overlapdifferential.

FIG. 4C illustrates an exemplary step for developing a document imagewith recognizable specified features through use of overlapdifferential.

FIG. 4D illustrates an exemplary step for developing a document imagewith recognizable specified features through use of overlapdifferential.

FIG. 4E illustrates an exemplary step for developing a document imagewith recognizable specified features through use of overlapdifferential.

FIG. 5A illustrates an exemplary step for manually selecting an area offocus utilizing a BIDA image capture system on a mobile device withimage capturing capabilities.

FIG. 5B illustrates an exemplary step for manually selecting an area offocus utilizing a BIDA image capture system on a mobile device withimage capturing capabilities.

FIG. 5C illustrates an exemplary step for manually selecting an area offocus utilizing a BIDA image capture system on a mobile device withimage capturing capabilities.

FIG. 5D illustrates an exemplary step for manually selecting an area offocus utilizing a BIDA image capture system on a mobile device withimage capturing capabilities.

FIG. 5E illustrates an exemplary step for manually selecting an area offocus utilizing a BIDA image capture system on a mobile device withimage capturing capabilities.

FIG. 6 illustrates exemplary method steps for utilizing a BIDA imagecapture system on an electronic device with image capturing and networkconnection capabilities.

FIG. 7A illustrates exemplary method steps for performing a BIDA imagecapture process.

FIG. 7B illustrates exemplary method steps for performing a BIDA imagecapture process.

FIG. 8 illustrates additional aspects of apparatus that may be used insome implementations of the present invention.

FIG. 9 illustrates a block diagram of a mobile device that may be usedin some implementations of the present invention.

DETAILED DESCRIPTION

The present invention relates generally to a preliminary imageprocessing system that analyzes a set of image data and characterizesthe quality of the set of image data. In some preferred embodiments, thepreliminary image processing system is incorporated into a mobile devicesuch as a smart phone and includes image capture hardware and softwareand is further capable to communicate via a distributed communicationsnetwork to a storage device, such as, for example a cloud document typestorage apparatus. According to the present invention a set of imagedata is captured and processed to determine a threshold of image quality(TOIQ) prior to transmission of the set of image data to the storageapparatus. A threshold of image quality may include, for example, userdefined parameters for a percent of success of optical characterrecognition, object image recognition, a potential for test characterthreshold, biometric recognition or other standard.

Glossary

-   As used herein the following terms will have the following    associated meaning:-   Bounded Image Data Analysis (BIDA): as used herein refers to an    analysis of a captured image based on a predefined set of criteria,    wherein a local system processes the analysis. The system-defined    analysis comprises a preliminary feature quality analysis that may    assess whether feature recognition may be possible with an    acceptable threshold of image quality during continued remote    processing.-   Feature Recognition: as used herein refers to a capability to    isolate and identify specified features on an image, wherein the    specified feature may depend on a type or category of image data.-   Examples of Feature Recognition may include: physical object,    alphanumeric text recognition, barcode recognition, hash code    recognition, feature recognition, biometric recognition, or other    input recognition that may be captured as a two dimensional (2D)    image on a three dimensional (3D) image derived from one of many    known techniques of imaging.-   Specified Feature: as used herein refers to a component of an image    that when processed may identify and categorize an image. For    example, for a contract document, the specified feature may comprise    alphanumeric characters and symbols. For a photograph, the specified    features may comprise objects, including, for example, an article,    person, device, structure, or combinations thereof. For some images,    such as a driver's license, the specified features may comprise a    combination of characters and objects utilized in facial    recognition.-   Local System: as used herein refers to a device or system of devices    that may process the BIDA without requiring logical communication    with a network.-   Overlap Differential: as used herein refers to an image created    through an amalgamation of multiple captured images of the same    document, wherein feature recognition may not be possible on an    isolated captured image but may be possible on the amalgamated    image.-   Authenticated: as used herein refers to a state where the image    capture and remote access rights have been verified by a predefined    verification mechanism, including, for example, time stamp, user    login, unique ID of image capture device, such as a IMEI, voice    recognition, geolocation, fingerprint, or face recognition.-   Continued Remote Process: as used herein refers to a process    separate from the BIDA, wherein the separate process occurs after    receipt of an image from the local system over a network.-   Field of View: as used herein refers to the viewable boundaries of    an image capturing device.-   Area of Focus: as used herein refers to a subset of boundaries    within the field of view where the image capturing device may be    capable of focusing text or images within the subset.-   Document Image: as used herein refers to a captured view of a    document, wherein the captured view may comprise at least one    specified feature.-   BIDA Image Capturing System: as used herein refers to an image    capturing device component of the local system, which may be    configured to capture images that may be compatible with the BIDA.    In some embodiments, the BIDA image capturing system may    automatically frame and focus the image capturing mechanism of the    electronic device. Alternatively, the BIDA image capture system may    comprise a manual mode, wherein a user may control the framing and    focusing of the image capturing mechanism.

Referring now to FIGS. 1A-1B, exemplary embodiments illustrate someimplementations of the present invention for engaging a BIDA on a localsystem, wherein the local system comprises at least an image-capturingmechanism and a processor on a local system 100 which may be anexemplary portable computing device, such as, for example a mobile smartphone. FIGS. 1A and 1B illustrate an exemplary embodiment of the BIDAcapture system where the system may access and review images storedwithin a photograph album or other such file storage on the local system100. FIG. 1A illustrates an image where the BIDA capture system mayrecognize specified features such as a line of text 121 on a documentimage 120. The user interface for file-based BIDA may include suchfeatures as an icon 140 to designate the local storage of imagescaptured in file-based BIDA.

An ability to review the images in the storage location such as iconsfor proceeding through the images in a first direction at 154 or areverse direction at 155. The image may be chosen at 165 or rejected at160 as non-limiting examples of a user interface. A particular documentimage 120 may be represented where specified features 121 may includetext that may be located where the text may have been successfullyscanned. On a successful detection of character text during BIDA, anotification 130 may be provided to the user, such as, for example, oneor more of: a vibration, a tone, a verification number certifyingacceptance of the image at a defined quality level, and acceptance icon,or other indicator. A verification may include, for example, auniversally unique identification number (UUID) that may be stored onone or both of the mobile device and the cloud document mechanism.

FIG. 1B illustrates a document image 125 where the BIDA capture systemmay not recognize a specified feature 126. In some embodiments, theremay still be a notification of some kind including one or more ofaudible, visual, or tactile indicator. In other embodiments, the absenceof a notification may be the indicator that the document image does notcomprise recognizable features and does not pass a BIDA capture test,such as meeting a threshold of image quality. In some embodiments, avisual notification 131 may highlight the specified feature 126 that maynot be recognizable at a certain distance from the camera, which mayallow a user to decide to move the camera orientation and/or adjustlighting before taking a new image capture of the document or to acceptthe document image despite one or more unrecognizable specifiedfeatures.

Referring now to FIGS. 2A and 2B, exemplary embodiments of the BIDAcapture system utilizing the image capturing mechanism on a local system200 for different configuration setup. FIG. 2A may illustrate a localsystem 200 where the BIDA capture system may not recognize a specifiedfeature 221 within a document 220. This may be due to various factorsincluding an inadequate focusing condition or the lack of textual datacapable of an efficient OCR in the image captured. For example, asillustrated, the document 220 may be within the field of view 210 of thecapture system but may be beyond the area of focus 211. Accordingly, theBIDA capture system may not notify the user of a successful scan,wherein the scan meets defined criteria.

FIG. 2B may illustrate an image capture where the BIDA capture system onthe local system 200 may recognize the specified feature 221 on thedocument 220 and provide a notification 230 to the user, such as one ormore of a visual, tactile, or audible indication. For example, thenotification 230 may comprise a vibration or audio laying of a .wavfile, which may articulate or announce the condition of the capturedimage. In some examples, an alternate notification may occur if featurerecognition fails during a BIDA of a captured image.

In some embodiments, the BIDA capture system may engage the targeteddocument once the local system 200 captures the image. In someembodiments, the BIDA capture system may perform a continuous scanprocess when the image capturing device is actively capturing images. Insuch embodiments, the BIDA capture system may control the capturingmechanism and trigger the capture when text is recognized according to athreshold of image quality where a PFQA is successful. In still furtherembodiments, the BIDA may occur multiple times. For example, the BIDAmay occur while the camera is active and then again after the image hasbeen captured. Such combinations may allow the user to consistentlycapture readable images combined with a secondary confirmation scan. Insome embodiments, the image capture device may capture a series ofimages in quick succession or extract images of a document from a videocapture. The specific settings may be based on the image capturingcapabilities of the local and/or user preferences.

Referring now to FIGS. 3A-3F, exemplary steps for utilizing a localsystem with BIDA capabilities and uploading a captured document areillustrated. FIG. 3A illustrates application icons, including a BIDAcapture system icon 370, and a camera icon 375 on a portable computingdevice 300, wherein a user 380 may select the BIDA capture system icon370. In some embodiments, at FIG. 3B the BIDA capture system may prompta user to select between two image selection icons; a photograph albumicon 340, and a camera icon 345. The user 380 may click a camera icon345, which may initiate the image capturing mechanism on the mobiledevice, or the photograph album icon 340, which may access documentimages stored on the portable computing device 300.

At FIG. 3C, the user 380 may select a captured image 320 to send to asecure cloud based image storage facility over a network for continuedremote processing. In some exemplary embodiments, the user 380 may cyclethrough captured images in the folder by clicking directional arrows354, 355. The user 380 may confirm selection of the captured image 320by selecting the remote secure cloud based image storage 365 icon. Insome embodiments, at FIG. 3D, a user 380 may select the folder withinthe remote secure cloud based image storage to direct the storage of thecaptured image 320. In other embodiments, not shown, the BIDA may becapable of limited content analysis, wherein the local system maysuggest folders to place image files.

At FIG. 3E, the user 380 may be prompted to respond to an authenticationstep. In some exemplary embodiments, the authentication step may beprompt to connect a dongle or chip to the portable computing device 300,which may be used to confirm the identity of the user 380. In alternateembodiments, the image capturing mechanism may flip and capture a facialimage of the user 380. In still further embodiments, the authenticationstep may comprise a voice recognition, and the portable computing device300 may be programmed to recognize specific words spoken by the user380. The authentication step may comprise a fingerprint recognition orother biometric as available on some smart mobile devices. At FIG. 3F,the local system may transmit the captured image 320 over a network 390to a remote secure cloud based image storage 395 for continued remoteprocessing.

Referring now to FIGS. 4A-4E, exemplary steps are illustrated fordeveloping a document image with recognizable specified features throughuse of overlap differential. At FIG. 4A, a BIDA image capturing device400 may capture an image of a document 420 comprising three specifiedfeature segments 421, 422, 423. The document 420 may be placed withinthe field of view 410 of the BIDA image capturing device but out ofrange of the area of focus 411. This step may allow the local system torecognize the boundaries of the document 420 but may not allow forspecified feature 421, 422, 423 recognition.

At steps 4B-4D, the BIDA image capturing device 400 may capture multipleimages of the document 420, wherein the recognizable specified features421, 422, 423 may be different at each step. The images captured at4B-4D capture focused and recognizable features 421, 422, 423,respectively. At 4E, the local system may process the multiple imagesutilizing overlap differential, which may develop a single documentimage 424 comprising completely recognizable features 421, 422, 423. TheBIDA image capturing device 400 may present the amalgamated documentimage 424 and notify 430 the user that the document image 424 comprisesrecognizable specified features. Embodiments may include notification ofrecognizable specified features across an entire document or a portionthereof.

In some exemplary embodiments, the images captured in FIGS. 4A-4D may becaptured through an automated series of image capture. In some alternateembodiments, the images captured may be extracted from a video segmentfilming the document. In some embodiments, the user may select multipleimages and confirm that the images comprise a single document. In someexemplary embodiments, the BIDA may determine whether multiple imagescomprise a single document or multiple documents. In some suchembodiments, the BIDA may identify the borders of a document andseparate multiple documents from a single captured image. Thisembodiment may be particularly useful when the BIDA may extract thedocuments from a video segment filming one or more documents.

In BIDA for more complex images, such as three-dimensional objectrecognition, known techniques, such as simultaneous localization andmapping (SLAM), may be implemented in part within the local system andcompleted through remote continued processing.

Referring now to FIGS. 5A-5E, exemplary steps for manually selecting anarea of focus are illustrated. At 5A, a BIDA image capturing device 500may capture an image of a document 520 comprising three specifiedfeature segments 521, 522, 523. The document 520 may be placed withinthe field of view 510 of the BIDA image capturing device but out ofrange of the area of focus 515. This step may allow the local system torecognize the boundaries of the document 520 but may not allow forspecified feature 521, 522, 523 recognition. At 5B, the BIDA imagecapturing device 500 may present the document image 524 to the user 580.In other embodiments, multiple documents may be located within the fieldof view 510, and the local system may be capable of identifying multipleborders and isolating the documents.

In some embodiments at 5B, the user 580 may select a particularspecified feature 523 deemed significant to the document, wherein thedocument image may be adequate as long as the selected specified featuremay be recognizable. For example, a receipt may comprise the amounts anditems of the sale, but it may also comprise an intricate logo for thestore. A user may determine that the logo may not be a crucial piece ofinformation for that receipt. Accordingly, the user may specify that thenecessary specified features comprise the bottom text 523.

At step 5C, the BIDA image capturing device 500 may capture an image orbe capable of capturing an image wherein unselected specified features521 may be recognizable but the selected specified features 522 may beout of the field of view 510. At step 5D, the BIDA image capturingdevice 500 may capture or be in a position to capture a document image,wherein the document image may comprise recognizable specified features522 that the user 580 may have selected at step 5B. In some embodiments,the feature recognition may trigger a notification 530 from the imagecapturing device. At step 5E, the document image 525 may be presented tothe user 580, who may select the document image 525 for transmission toa secure cloud based image storage for continued remote processing.

Referring now to FIG. 6, an exemplary process 600 for utilizing a BIDAcapture system may be illustrated. At 605, a user may activate the BIDAcapture system. In some embodiments, the function may occur on a mobiledevice. In some embodiments, a user may manually activate the BIDAcapture system, such as illustrated in FIG. 1. In some alternativeembodiments, a user may activate the BIDA image capture systemindirectly by operating a mobile device comprising the BIDA imagecapture system. For example, a digital camera may automatically initiatethe BIDA image capture system when a user is taking photographs orotherwise capturing image data.

Steps 610, 620 illustrate method steps for utilizing the BIDA imagecapture system on existing images stored on the mobile device or on amemory card. At 610, the user may select an image file folder, and at620 may select an image. Steps 615, 625 illustrate method steps forutilizing the BIDA image capture system with an image capturingfunction. At 615, the user may select the image capture function, and at620, the user may capture an image. At 630, the user may wait for aconfirmation indication from the BIDA image capture system. As describedin reference to FIG. 1A and FIG. 1B, the BIDA image capture system mayscan a document before the user captures its image at 625.

At 635, a user may receive no response or a negative response from theBIDA image capture system, which may prompt the user to restart theimage selection process at 610, 620 or image capturing process 615, 625.Alternatively, at 640, a user may receive a confirmation indication. Insome embodiments, a successfully recognized document may beautomatically stored without requiring further input from a user. Inother embodiments, the user may accept and store the document image at645 or may reject the document at 650. When a document is stored at 645,the application may communicate with a storage system such as a serverand a control application on the storage system, at 655, may verify theidentification of the user associated with the image capture device orthe application communicating the image data. In the process ofverifying the image, further processing of the document image data mayoccur on the storage system for appropriate storage as a fully processeddigital document. At 660, the user may exit the application directly orindirectly.

Referring now to FIG. 7A, exemplary method steps of a BIDA image captureprocess 700A are illustrated. At 701, a BIDA image capture system maycapture a document image. In some embodiments, a user may select apreviously captured document image from a file folder, such as aphotograph album, stored within the electronic device. Some embodiments,such as illustrated in reference to FIGS. 1A and 1B, may allow a user tochoose to capture a new document image or select from a previouslycaptured document image.

At 702, the BIDA image capture system may identify the borders using oneor more of many well known techniques such as contrast edge detection ofthe documents within the view, and at 703 the BIDA image capture systemmay identify potential characters. The steps at 702, 703 may provideframing and orientation cues, which may allow for more reliable BIDAcapture. As described in reference to FIG. 1A and FIG. 1B, in someembodiments, the BIDA image capture system may control the imagecapturing mechanism of the electronic device, and such framing andfocusing may be automatically performed, without requiring further inputfrom the user. At 704, the BIDA image capture system may analyze atleast a portion of the potential characters identified at 703.

At 705, the results of the analysis may be transmitted to a notificationmechanism, wherein the notification mechanism for example an OCRprocessing algorithm which, may discern between and fail result and asuccess result based on predefined parameters. In some embodiments, asuccess result may comprise a complete text character recognition of atleast a portion of the identified potential text characters, which maycomprise a portion of the document image text characters. For example,the BIDA image capture system may identify specific text characters,such as those considered more difficult to recognize or more likely toallow full word recognition.

In other embodiments, a success result may comprise a partial textcharacter recognition of at least a portion of the identified potentialtext characters, wherein predefined portions of text characters may berecognized. For example, analyzing a select portion of a string of textcharacters may be sufficient to establish legibility of a documentimage. In still further embodiments, the predefined parameters may bebased on a combination of recognition factors.

In some embodiments, a preliminary optical character recognition systemmay transmit a fail result at 706, the notification mechanism mayoptionally notify the user at 707. At 708, a fail result may trigger aprompt to the user to select or capture an alternate document.Alternatively, in some embodiments the preliminary optical characterrecognition system may transmit a success result 709, and thenotification mechanism may notify the user at 710. As discussed above, anotification may include one or more of audible, visual, or tactilenotifiers, such as a vibration. In some embodiments, the notificationmay only occur with a success result at 710.

Where the preliminary optical character recognition system may transmita success result at 709, the preliminary optical character recognitionsystem may optionally prompt the user to accept or reject the documentat 711, which may confirm that the user intended to select and/orcapture the particularly document image. At 712, the preliminary opticalcharacter recognition system may verify access authorization of theelectronic device, and at 713, the preliminary optical characterrecognition system may optionally prompt the user to provide responsesauthentication or security inquiries. Such security measures may besignificant where the user intends to transmit the document image to asecure external server storage, such as a cloud based image storagesystem.

In some embodiments, at 714, a preliminary optical character recognitionsystem may be capable of categorizing the document image. For example,based on a predefined classification system and recognition parameters,the preliminary optical character recognition system may be able toanalyze identified characters and identify key words, phrases, logos,patterns of words, proximity of words to each other, or other criteria.The analysis may be used to discern between a purchased item, a page oftext, a face, an object, an official government document, a receipt, adeed, a birth certificate, and a tax document. In some embodiments, thepreliminary optical character recognition system may prompt the user toconfirm or reject the categorization, allowing the user to specify thedestination file folder or classification of the document image. At 715,the preliminary optical character recognition system may transmit thedocument image file and related OCR results to an external server,wherein the external server may be capable of a complete opticalcharacter recognition analysis. In some embodiments, such as a mobilephone or tablet, the electronic device may be capable of wirelesslyconnecting to the external server and transmitting the data.Alternately, in other embodiments such as with a digital camera, theelectronic device may require a hard connection to a secondary device,such as a laptop computer, capable of connecting to a network system.

Referring now to FIG. 7B, similar to the above example relating to anobject containing text, exemplary method steps of a BIDA object imagecapture process 700B are illustrated. At 721, a BIDA image capturesystem may capture an object image. In some embodiments, a user mayselect a previously captured object image from a file folder, such as aphotograph album, stored within the electronic device. Some embodiments,such as illustrated in FIG. 1A and FIG. 1B, may allow a user to chooseto capture a new object image or select from a previously capturedobject image.

At 722, the BIDA image capture system may identify the borders ofobjects within the view, and at 723 the BIDA image capture system mayidentify potential types of objects using one or more of many well knowntechniques such as contrast edge detection of the documents within theview. The steps at 722, 723 may provide framing and orientation cues, orcontrast which may allow for more reliable BIDA capture. As described inreference to FIG. 1A and FIG. 1B, in some embodiments, the BIDA imagecapture system may control the image capturing mechanism of theelectronic device, and such framing and focusing may be automaticallyperformed, without requiring further input from the user. At 724, theBIDA image capture system may analyze at least a portion of thepotential objects identified at 723.

At 725, the results of the analysis may be transmitted to a notificationmechanism, wherein the notification mechanism may discern between andfail result and a success result based on predefined parameters. In someembodiments, a success result may comprise a complete object recognitionof at least a portion of the identified potential object, which maycomprise a portion of the object image. For example, the BIDA imagecapture system may identify specific object characteristics, such asthose considered more difficult to recognize or more likely to allowfull object recognition.

In other embodiments, a success result may comprise a partial objectrecognition of at least a portion of the identified potential object,wherein predefined portions of object may be recognized. For example,analyzing a select portion of a string of objects may be sufficient toestablish recognition of an object within an image. In still furtherembodiments, the predefined parameters may be based on a combination ofrecognition factors.

In some embodiments, a preliminary object recognition system maytransmit a fail result at 726, the notification mechanism may optionallynotify the user at 727. At 728, a fail result may trigger a prompt tothe user to select or capture an alternate object. Alternatively, insome embodiments the preliminary object recognition system may transmita success result 729, and the notification mechanism may notify the userat 730. As discussed above, a notification may include one or more ofaudible, visual, or tactile notifiers, such as a vibration. In someembodiments, the notification may only occur with a success result at730.

Where the preliminary object recognition system may transmit a successresult at 729, the preliminary object recognition system may prompt theuser to accept or reject the object at 731, which may confirm that theuser intended to select and/or capture the particularly object image. At732, the preliminary object recognition system may verify accessauthorization of the electronic device, and at 733, the preliminaryobject recognition system may prompt the user to provide responses tosecurity inquiries. Such security measures may be significant where theuser intends to transmit the object image to a secure external serverstorage, such as a cloud based image storage system.

In some embodiments, at 734, a preliminary object recognition system maybe capable of categorizing the object image. For example, based on apredefined classification system and recognition parameters, thepreliminary object recognition system may be able to analyze identifiedobject characteristics and identify key words, phrases, logos, patterns,proximity of patterns to each other, or other criteria. At 735, thepreliminary object recognition system may transmit the object image fileand related recognition results to an external server, wherein theexternal server may be capable of a complete object recognition orcharacter recognition analysis. In some embodiments, such as a mobilephone or tablet, the electronic device may be capable of wirelesslyconnecting to the external server and transmitting the data.Alternately, in other embodiments such as with a digital camera, theelectronic device may require a hard connection to a secondary device,such as a laptop computer, capable of connecting to a network system.

Referring now to FIG. 8, additional aspects of controller hardware whichmay be included as computer hardware, useful for implementing thepresent invention may be illustrated as a block diagram that may includea controller 850 upon which an embodiment of the invention may beimplemented. Controller 850 may include a bus 852 or other communicationmechanism for communicating information, and a processor 854 coupledwith bus 852 for processing information.

Controller 850 may also include a main memory 856, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 852for storing information and instructions to be executed by processor854. Main memory 856 may also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 854. Controller 850 may further include a readonly memory (ROM) 858 or other static storage device 860.

Controller 850 may be coupled via bus 852 to a display 862, such as acathode ray tube (CRT) or liquid crystal display (LCD), for displayinginformation to a computer user. An input device 864, includingalphanumeric and other keys, or modes of input, such as, for example, amicrophone and a radio frequency device such as Bluetooth, may becoupled to bus 852 for communicating information and command selectionsto processor 854. Another type of user input device may be a cursorcontrol 866, such as a mouse, a trackball, a touchpad, touchscreen, orcursor direction keys for communicating direction information andcommand selections to processor 854 and for controlling cursor movementon display 862. This input device may typically have two or threedegrees of freedom in two axes, a first axis (e.g., x) and a second axis(e.g., y), that allows the device to specify positions in a plane orstereo cameras that process and provide a third axis of input.

Some embodiments of the invention may be related to the use ofcontroller 850 for setting operational parameters. According to oneembodiment of the invention, control parameters may be defined andmanaged by controller 850 in response to processor 854 executing one ormore sequences of one or more instructions contained in main memory 856.Such instructions may be read into main memory 856 from anothercomputer-readable medium, such as storage device 860. Execution of thesequences of instructions contained in main memory 856 causes processor854 to perform the process steps described herein. In alternativeembodiments, hard-wired circuitry may be used in place of or incombination with software instructions to implement the invention. Thus,embodiments of the invention are not limited to any specific combinationof hardware circuitry and software.

The term “computer-readable medium” as used herein may refer to anymedium that participates in providing instructions to processor 854 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media.Non-volatile media includes, for example, solid state devices (SSD) ormagnetic disks, such as storage device 860. Volatile media may includedynamic memory, such as main memory 856. Transmission media includescoaxial cables, copper wire and fiber optics, including the wires thatcomprise bus 852. Transmission media may also take the form of infraredand radio frequency transmissions, acoustic or light waves, such asthose generated during radio wave and infrared data communications.

Common forms of computer-readable media may include, for example, amemory stick, hard disk or any other magnetic medium, a CD-ROM, anyother optical medium, a RAM, a PROM, and EPROM, a FLASH-EPROM, any othermemory chip or cartridge, a carrier wave as described hereinafter, orany other medium from which a computer can read.

Various forms of computer readable media may be involved in carrying oneor more sequences of one or more instructions to processor 854 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over adistributed network such as the Internet. A communication device mayreceive the data on the telephone line and use an infrared transmitterto convert the data to an infrared signal. An infrared detector mayreceive the data carried in the infrared signal and appropriatecircuitry can place the data on bus 852. Bus 852 may carry the data, orotherwise be in logical communication to the main memory 856, from whichprocessor 854 retrieves and executes the instructions. The instructionsreceived by main memory 856 may optionally be stored on storage device860 either before or after execution by processor 854.

Controller 850 may also include a communication interface 869 coupled tobus 852. Communication interface 869 provides a two-way datacommunication coupling to a network link 870 that may be connected to alocal network 872. For example, communication interface 869 may operateaccording to the internet protocol. As another example, communicationinterface 869 may be a local area network (LAN) card a datacommunication connection to a compatible LAN.

Network link 870 may typically provide data communication through one ormore networks to other data devices. For example, network link 870 mayprovide a connection through local network 872 to a host computer 874 orto data equipment operated by an Internet Service Provider (ISP) 876. Awireless links may also be implemented. ISP 876 in turn may provide datacommunication services through the worldwide packet data communicationnetwork now commonly referred to as the “Internet” 879. Local network872 and Internet 879 may both use electrical, electromagnetic or opticalsignals that carry digital data streams. The signals may be transmittedthrough the various networks and the signals on the network link 870 andthrough communication interface 869, which carry the digital data to andfrom controller 850 are exemplary forms of carrier waves transportingthe information.

In some embodiments, Controller 850 may send messages and receive data,including program code, through the network(s), network link 870 andcommunication interface 869. In the Internet example, a server 890 mighttransmit a requested code for an application program through Internet879, ISP 876, local network 872 and communication interface 869.

Processor 854 may execute the received code as it is received, and/orstored in storage device 860, or other non-volatile storage for laterexecution. Some exemplary controllers 850 may include a personal digitalassistant, a mobile phone, a smart phone, a tablet, a netbook, anotebook computer, a laptop computer, a terminal, a kiosk or other typeof automated apparatus. Additional exemplary devices may include anydevice with a processor executing programmable commands to accomplishthe steps described herein.

FIG. 9 is a block diagram of some embodiments of a network access devicethat may include a mobile device 902. The mobile device 902 comprises anoptical capture device 908 to capture an image and convert it tomachine-compatible data, and an optical path 906, typically a lens, anaperture or an image conduit to convey the image from the rendereddocument to the optical capture device 908. The optical capture device908 may incorporate a Charge-Coupled Device (CCD), a Complementary MetalOxide Semiconductor (CMOS) imaging device, or an optical sensor ofanother type.

A microphone 910 and associated circuitry may convert the sound of theenvironment, including spoken words, into machine-compatible signals.Input facilities 914 exist in the form of buttons, scroll-wheels orother tactile sensors such as touch-pads. In some embodiments, inputfacilities 914 may include a touchscreen display.

Visual feedback-to the user is possible through a visual display,touchscreen display, or indicator lights. Audible feedback 93.4 may comefrom a loudspeaker or other audio transducer. Tactile feedback may comefrom a vibrate module 936.

A motion sensor 938 and associated circuity convert the motion of themobile device 902 into machine-compatible signals. The motion sensor 938may comprise an accelerometer, which may be used to sense measurablephysical acceleration, orientation, vibration, and other movements. Insome embodiments the motion sensor 938 may include a gyroscope or otherdevice to sense different motions.

A location sensor 940 and associated circuitry may be used to determinethe location of the device. The location sensor 940 may detect GlobalPosition System (GPS) radio signals from satellites or may also useassisted GPS where the mobile device may use a cellular network todecrease the time necessary to determine location. In some embodiments,the location sensor 940 may use radio waves to determine the distancefrom known radio sources such as cellular towers to determine thelocation of the mobile device 902. In some embodiments these radiosignals may be used in addition to GPS.

The mobile device 902 comprises logic 926 to interact with the variousother components, possibly processing the received signals intodifferent formats and/or interpretations. Logic 926 may be operable toread and write data and program instructions stored in associatedstorage 930 such as RAM, ROM, flash, or other suitable memory. It mayread a time signal from the clock unit 928. In some embodiments, themobile device 902 may have an on-board power supply 932. In otherembodiments, the mobile device 902 may be powered from a tetheredconnection to another device, such as a Universal Serial Bus (USB)connection.

The mobile device 902 also includes a network interface 916 tocommunicate data to a network and/or an associated computing device.Network interface 916 may provide two-way data communication. Forexample, network interface 916 may operate according to the internetprotocol. As another example, network interface 916 may be a local areanetwork (LAN) card allowing a data communication connection to acompatible LAN. As another example, network interface 916 may be acellular antennae and associated circuitry which may allow the mobiledevice to communicate over standard wireless data communicationnetworks. In some implementations, network interface 916 may include aUniversal Serial Bus (USB) to supply power or transmit data. In someembodiments other wireless links may also be implemented.

As an example of one use of mobile device 902, a reader may scan sometext from a newspaper article with mobile device 902. The text isscanned as a bit-mapped image via the optical capture device 908. Logic926 causes the bit-mapped image to be stored in memory 930 with anassociated time-stamp read from the clock unit 928. Logic 926 may alsoperform optical character recognition (OCR) or other post-scanprocessing on the bit-mapped image to convert it to text. Logic 926 mayoptionally extract a signature from the image, for example by performinga convolution-like process to locate repeating occurrences ofcharacters, symbols or objects, and determine the distance or number ofother characters, symbols, or objects between these repeated elements.The reader may then upload the bit-mapped image (or text or othersignature, if post-scan processing has been performed by logic 926) toan associated computer via network interface 916.

As an example of another use of mobile device 902, a reader may capturesome text from an article as an audio file by using microphone 99 as anacoustic capture port. Logic 926 causes audio file to be stored inmemory 928. Logic 926 may also perform voice recognition or otherpost-scan processing on the audio file to convert it to text. As above,the reader may then upload the audio file (or text produced by post-scanprocessing performed by logic 926) to an associated computer via networkinterface 916.

A controller may include one or more of: personal computers, laptops,pad devices, mobile phone devices and workstations located locally or atremote locations, but in communication with the controller. Systemapparatus may include digital electronic circuitry included withincomputer hardware, firmware, software, or in combinations thereof.Additionally, aspects of the invention may be implemented manually.

Apparatus of the invention may be implemented in a computer programproduct tangibly embodied in a machine-readable storage device forexecution by a programmable processor and method actions can beperformed by a programmable processor executing a program ofinstructions to perform functions of the invention by operating on inputdata and generating output. The present invention may be implementedadvantageously in one or more computer programs that are executable on aprogrammable system including at least one programmable processorcoupled to receive data and instructions from, and to transmit data andinstructions to, a data storage system, at least one input device, andat least one output device. Each computer program may be implemented ina high-level procedural or object oriented programming language, or inassembly or machine language if desired, and in any case, the languagecan be a compiled or interpreted language. Suitable processors mayinclude, by way of example, both general and special purposemicroprocessors.

Generally, a processor may receive instructions and data from aread-only memory and/or a random access memory. Generally, a computermay include one or more mass storage devices for storing data files;such devices include Solid State Disk (SSD), magnetic disks, such asinternal hard disks and removable disks magneto-optical disks andoptical disks. Storage devices suitable for tangibly embodying computerprogram instructions and data include all forms of non-volatile memory,including, by way of example, semiconductor memory devices, such asEPROM, EEPROM, and flash memory devices; magnetic disks such as,internal hard disks and removable disks; magneto-optical disks; and CDROM disks may be included. Any of the foregoing may be supplemented by,or incorporated in, ASICs (application-specific integrated circuits).

In some embodiments, implementation of the features of the presentinvention may be accomplished via digital computer utilizing uniquelydefined controlling logic, wherein the controller includes an integratednetwork between and among the various participants in ProcessInstruments.

The specific hardware configuration used may not be particularlycritical, as long as the processing power is adequate in terms ofmemory, information updating, order execution, redemption and issuance.Any number of commercially available database engines may allow forsubstantial account coverage and expansion. The controlling logic mayuse a language and compiler consistent with that on a CPU included inthe controller. These selections may be set according to per sewell-known conventions in the software community.

The present invention is described herein with reference to blockdiagrams and functional illustrations of methods and apparatus toimplement various aspects of the present invention. It is understoodthat each block of the block diagrams or operational illustration orfunction represented, and combinations of blocks in the block diagramsor operational or functional illustrations, may be implemented byautomated apparatus, such as analog or digital hardware and computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, cellular device, smart device, ASIC, or other programmabledata processing apparatus, such that the instructions, which execute viathe processor of the computer or other programmable data processingapparatus, implements the functions/acts specified in the block diagramsor operational block or blocks. In some implementations, the functionsor method steps described in relation to the blocks or functionalrepresentations may occur in an order other than the order noted ordescribed herein, For example, blocks or functional representationsshown in a succession may be executed substantially concurrently or theblocks in an alternate order, depending upon a specific implementationof the present invention. It is therefore understood that unlessotherwise specifically noted and thereby limited, the discussion here ispresented in an order to facilitate enablement and understanding and isnot meant to limit the invention disclosed.

CONCLUSION

A number of embodiments of the present invention have been described.While this specification contains many specific implementation details,there should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of the present invention.

Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented incombination in multiple embodiments separately or in any suitablesub-combination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous.

Moreover, the separation of various system components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order show, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous. Nevertheless, it will be understood thatvarious modifications may be made without departing from the spirit andscope of the claimed invention.

What is claimed is:
 1. An image feature processing apparatus comprising:an image capture device for capturing a set of image data; a preliminaryimage recognition mechanism operative to identify and analyze potentialcharacteristics included in the set of captured image data, whereinpreliminary image recognition comprises automated identification of afeature based upon at least partial recognition of the feature; and anotification mechanism configured to notify a user, without artificialdelay, of a result of the preliminary image recognition as compared topredefined parameters.
 2. The image feature processing apparatus ofclaim 1 wherein the preliminary image recognition mechanism providesoptical character recognition identifying potential text characters fromthe image data, wherein preliminary optical character recognitioncomprises one or both of: partial character recognition of potentialtext characters and character recognition of a portion of at least onetext character; and the notification mechanism notifies a user of apreliminary optical character recognition based on predefined parametersas specified by the user for a document type captured in the image data.3. The image feature processing apparatus of claim 1, wherein the imagecapture device is capable of connecting to cloud storage and processingserver.
 4. The image feature processing apparatus of claim 3, whereinthe image capture device comprises a mobile device.
 5. The image featureprocessing apparatus of claim 3, wherein the image capture devicecomprises a digital camera.
 6. The image feature processing apparatus ofclaim 3, wherein the preliminary image recognition mechanism isconfigured to identify potential text characters from a document imageand analyze the potential text characters with optical characterrecognition to provide a degree of character recognition.
 7. A method ofpreliminary optical character recognition and notification, the methodcomprising the method steps of: identifying at least one potential textcharacter from a captured document image, wherein an electronic devicewith image capture capabilities is configured to store the documentimage; analyzing the at least one potential text character withpreliminary optical character recognition, wherein the analyzing createsresults based on an ability to recognize at least one text character,and wherein the preliminary optical character recognition comprises oneor both a partial character recognition or a character recognition of aportion of the at least one text character; and transmitting the resultsto a notification device configured to notify a user of the resultsbased on predefined parameters of a fail result or a success result. 8.The method of claim 7 further comprising the method steps oftransmitting the captured document image to a cloud server for storageof the document image on the cloud server.
 9. The method of claim 8,wherein a transmission is based a result of analyzing wherein the resultcomprises a success result based upon recognition of one or both thepartial character recognition or the recognition of the portion of theat least one text character.
 10. The method of claim 9, furthercomprising the method steps of: transmitting the document image to anexternal server, wherein the external server is configured to identifythe potential text characters from the document image and analyze thepotential text characters with optical character recognition.
 11. Themethod of claim 7, wherein the notifying comprises at least one of anaudible notification, a visual notification, or a tactile notification.12. The method of claim 7 further comprising the method steps of:identifying borders of the document image; and recognizing anorientation of the at least one potential text character.
 13. The methodof claim 10 further comprising the method steps of: categorizing thedocument image based on predefined content criteria of the recognizedtext.
 14. The method of claim 10 further comprising the method steps of:prompting the user to accept or reject the document image.
 15. Themethod of claim 10 further comprising the method steps of prompting theuser to input authorization information, wherein the authorizationinformation is capable of granting access to a secure data storagesystem.
 16. The method of claim 7 wherein the notification of a failresults comprises prompting the user to select or capture an alternatedocument image.